16-899C ACRL Tetris Reinforcement Learner

Our approach to this problem was to use reinforcement learning with a function approximator to approximate the state value function [RSS98]. In our case, a +1 reward was given for every completed line, so that the value function would encode the long-term number of lines that is going to be completed by the algorithm. In order to achieve this, we extract features from the game state, and use gradient descent to update the parameters of our function approximator. Hence our approach is similar to Q-learning with a function approximator [RSS98], however contrary to Q-learning we learn only the state value function rather than the state-action value function. This is possible in our case since we can easily compute the exact immediate rewards and the exact next state of the game given the current state and the action executed by the agent. Learning the state value function is more practical since it can be represented with fewer parameters than the state-action value function, allowing our approach to learn faster.